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ABSTRACT 



A description is provided of the process used by a 



university institutional research office to respond to questions from 
external groups and the institutional community about quality, 
effectiveness, and accountability. The institutional research office 
incorporated external and internal reports, along with institutional 
offices' planning, accountability, and self-study documents, into a 
"live" document. The resulting document is "live," or "working," as 
it would be continuously revised as new studies and reports are done 
throughout the institution. This initiative is based on an 
institutional effectiveness planning and evaluation model that was 
developed using the Plan-Do-Check-Act configuration of total quality 
management. The process involved four main steps: (1) conducting a 
scan internally and externally for sources of "hot topics," (2) 
designing the model for the information to be collected, (3) 
developing a survey based on the information model to determine 
current documents that contain responsive information to the "hot 
topics," and (4) developing strategies to incorporate the process of 
collecting documents and using the information model as a viable 
resource. Characteristics which define a "hot topic" are identified. 
Appended is the document survey conducted by the institutional 
research office. (Contains 10 references -) (SW) 
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Answering the "Hot Topics" with 
One Comprehensive Document 

Abstract 

This paper describes the process used by one office of institutional research in an attempt 
to address the concerns of the external organizations and the institutional community about 
quality, effectiveness, and accountability through one comprehensive planning and evaluation 
document. External and internal reports, along with institutional offices' planning, accountability, 
and setf-study documents, j^ere incorporated into a "live" document that is based on an 
institutional effectiveness planning and evaluation model that was developed using the Plan-Do- 
Check-Act configuration of TQM. 
Introduction 

The basic premise of this paper is that the majority of the information that is needed to 
respond to external and internal concerns about quality, accountability, planning and evaluation 
are available within the educational institution or from external agencies. The design of this 
process is to be proactive, to anticipate needs of executive officers - the president/chancellor, the 
chief academic officer, the chief finance officer. The process is based on becoming more 
observant as to what is currently operational within the institution. The design is the find but what 
people are calking about; what are their concerns - their needs. It is set up as an environmental 
type scan, but not to i^abllsh the strengths and weaknesses, rather to discover what is being 
discussed externally and internally around the committee tables by the various stakeholders in the 
educational process or a particular institution. 

It seems that every year or so there is a new "hot topic," or a new demand, to justify what 
our educational institutions are doing, and we, ac institutional researchers and planners, are still 
working on the previous year's "hot topics". These "hot topics" appear to have caused educators, 
usually institutional researchers and planners, to spend an undue amount of time justifying the 
educational Institution to a variety of concerned organizations each with its own perceived 
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agenda. These organizations have challenged educational institutions to determine the "whos", 

"whats". Vhens". "wheres". and "V/hys" that w^re used to establish the existence of, and 

continuance of. our role as educators. A re.cent example is the Wingspread Report (1993). This 

report has five challenges for colleges and universities. They are: 

1. Evaluate yourselves against the questions in the attached "Self-Assessment 
Checklist." and to Cunmit yourself publicly to an institutional plan that 
builds on the strengths and remedies the deficiencies you identify.. 

^ 2. Define and publicly state your standards of entry and exit in terms of 
^ knowledge, skills, and abilities you expect from both applicar^fs and 

graduates, and to put in place measures to assure student and institutional 
attainment of those standards by a fixed date. 

3, Develop a curriculum that will assure all graduates— -our future citizens, 
employees, and leaders— the benefits of a liberal education. 

4. Assure that next year's entering students will graduate as individuals of 
character more sensitive to the needs of community, more competent to 
contribute to society, and more civil in habits of thought, speech, and 
action (Wingspread, 1993, p. 23). 

These reports and queries into what education is all about and how we are doing have 
forced us to try to justify that we. as educational institutions, are performing our role. To respond 
to these challenges and inquiries, one of several approaches can be taken; don1 respond to the 
reports/inquiries and continue on with the regular reporting operation of the institution; take the 
reports/inquiries to t\0i and respond to the assessment checklist or whatever is proposed; or 
deal with the Intent of the reports/inquiries and other 'hot topics" and demonstrate that the 
majority of the concerns are already addressed by the institution. 

This paper uses the latter approach, it is felt that there is no need to generate new 
Internal studies to address each "hot topic", that educational institutions have the majority of the 
answers. What is missing is a procedure by which ail the appropriate documents can be 
Identified and tied into one source. All the pieces of supporting information on how well the 
Institution Is doing its job may already be available, but no one person or one office knows all the 
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pieces. The process described in this paper can used at any institution, to locate, collect, and 
integrate all the different pieces; the planning documents, self-studies, evaluations, and 
assessments. The resulting document would be considered '*live," or Vorking,** as it would be 
continuously revised as new planning documents, new self-studies, etc. are done throughout the 
institution. The end result, hopefully, will be a document that contains the information to respond 
to the many external and internal challenges for today's institution. 
Process ^ 

The process used at this institution consisted for four main steps. The first was to conduct 
a scan, both internal and external for sources of "hot topics". The second step was to design the 
model for the information to be collected, and, third, based on the information model, develop a 
survey that would ascertain current documents which would contain responsive information to "hot 
topics'. The last step was to develop strategies to incorporate the process of collecting 
documents and using the infomiation model as a viable resource into the organizational culture. 

The first step is to discover what makes a topic "hot" and where to locate sources to be 
scanned for future "hot topics". "Hot topics" have been the subject of numerous fora, especially 
as they relate to new and emerging issues. But, what is a "hot topic" today can quickly become 
passe. So, how does an Institutional research^offlce identify the "hot topics", gather infomiation 
about them in a usefiiiWnanner, and, most importantly, disseminate that information in a fashion 
which allows for quick/easy access, and the degree of flexibility to change v/hat is "hot" with 
relative ease? 

A proactive Institutional research office should take the initiative in detennining what 
constitutes a "hot topic." In these authors* joint opinion, a "hot topic" is identified by the following 
characteristics: 

A. Legislated OegaO requirement 

A regulatory requirement exists from a governing body (federal/state or national/province) 
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outside the institution which legally imposes a mandate on the institution with sanctions for 
non-compliance. Generally, these are numerous legislative discussion over time which 
can be "early alert" signals that a "hot topic" is gestating. This type of mandate could also 
be originated by an accrediting agency (regional, profession, or national) with the 
■institutional effectiveness" "hot topic" of the Southern Association of Colleges and Schools 
(SACS) accreditation being a prime example in recent years. Mandated accountability 
standards are another example whereby legislated accountability results. _ 

B. Reach 

Extent of "reach/ or "market penetration," are operative marketing research terms which 
can be used to define the number of institutions affected/involved In the "hot topic" issue. 
An example of this factor exists in the NCAA where equity of women's sports becomes an 
issue. This factor has two major components: (1) size in frequency; and (2) geographical 
dispersion. In other words, the greater the frequency of institutions and the greater the 
geographical dispersion, the greater the "reach". 

C. Tit* with current values 

The extent to which existing societal/institutional values determines its "hotness." For 
example, in the United States, total quality management (TQM) and continuous quality 
improvement,^^^!) coincide with the current values concerning effectiveness and 
efficiency and the concept of increasing quality is "in vogue." Tlierefore, TQM and CQl 
have become "hot topics" (evidenced in the literature and in recent conferences/fora). 

D. Perception of Benefit 

The extent to which a concept/approach Is perceived as helpful in dealing with constraints 
on an institution determines degree of "hotness," 1. e., the greater the perception of benefit, 
the greater the "hotness," or urgency of the topic. If CQl is viewed as a way of optimizing 
the efficient and effective use of scarce resources, it becomes a "hot topic." 



ERLC 



V 



t 



Here are some sources to assist in locating and responding to your institution's "hot topics:" 

1. External 

a. National Level 

1) Federal government agencies, eg.. DOE. NCES. OERI 

2) Professional associations 

3) Professional/Discipline accreditation agencies 

b. Regional Level 

1) Accreditation agencies 

2) SREB. etc. 

c. State Level 

1) DOE. Boards of Higher Education. Community Colleges, 
Regents. Vocational Education 

2) Government agencies, legislature 

d. Local-Regional County Level 

1) Planning Boards 

2) Chambers of Commerce 

2. Internal 

a. Academic Affairs 

1) Colleges 

2) Departments 

3) Centers, Institutes 

4) Advisory Groups 

b. Non-academic Departments 

1) Administration & Finance Departments 

2) Student Affairs Departments/Student Government 

3) EEO Office 

4) Auditor Office 

5) Athletic Department 

6) Alumni Affairs 

7) Development Office 

8) OIR/evaluatlon/planning/effectiveness offices 

3. Media 

a. Print 

Electronic 



The next step Is to develop a design of the information to be collected. This step creates 
the foundation of the live" document, the process used to gather the responses as to what the 
Institution has done, is doing, and will do relative to a particular topic. The basis for the "live- 
document begins with developing an institutional effectiveness, planning and evaluation model. 
The model used In this paper was developed using the Plan Do Check Ad model for Total Quality 
Management In response to a regional accreditation (Figure 1). 
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Figure 1. 

PLAN-DO-CHECK-ACT CYCLE 
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/ ACT 


PLAN \ 


\ CHECK 


DO / 



There are four areas in the TQM mode!; Plan, Do, Check and Act. The first area. Plan, 
requires a statement of mission/purpose. Before you head out on any road you need to know 
where you are going or else you won't know when you have gotten there. Once the mission has 
be established, goals and objectives are written to verify the achievement of the mission, and the 
planning of the directions to be taken to help meet the goals and objectives are established 
(Figure 2). For purposes of illustration examples from a Florida state university are used in all the 
figures. 

mgure 2. 

PLAN CYCLE 



Statement of SUS M.sitr Phm 




The second area, Do. requires the Implementation of the operational plans that were 



designed to achieve the goals, objectives and hence the mission of the institution (Figure 3). 



Figure 3. 

DO CYCLE 




The third area, Check, dictates the necessity to evaluate what has been done to verify the 
meeting of the goals and objectives and the processes used to achieve them, and to evaluate the 
different components within the institution (Figure 4). 

Figure 4. 



CHECK CYCLE 
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The final area. Act. refers to the actual use of what was discovered in your planning and 
analysis to revise/complement the mission statement (Figure 5). 

Figure S. 

ACT CYCLE 




From this total framework one document can be developed which will assist in responding 
to concerns about quality, accountability, and effectiveness (Figure 6), 

Figure 6. 
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The next concern is how does one discover the different components which will fill in the 
various circles in the institutional effectiveness, planning and evaluation model. Beyond what was 
discovered in the "hot topics" scaa, there are two ways to establish networks that may supply the 
answers. The first is to develop the interpersonal relationships with the educational community. 
Don*t take it for granted that because you have been at a particular institution for a number of 
years that you don't need to work on re-establishing your networks with current administrative and 
academic committees/task forces/Internet distribution lists. Identify chairs of these working 
groups, identify what their mission is, discover what each one has done or is doing, offer your 
assistance at brainstomiing sessions, at points in time when they may be developing a survey. 
The second source is to sun/ey down to the department level exactly what each area has In the 
way of documents on planning, accountability, assessment, evaluation from the perspective of It 
being an input, outcome, or process. In the survey identify who is responsible for the document, 
what is entails, ie. format, methodology, results, procedures for using results and what changes 
occu red because of the resuits, where it is sent, what it is used for, and how often does the 
document get updated, and when. In the initial survey, limit the documents to within the past 
three years and what is scheduled for the next year. These elements comprise the final "live* 
document. It is recommended that from the materials collected that an executive summaty t>e 
created which would foference more specific infomnation if desired by the reader (Figure 7). 

EXECUTIVE SUMMARY 
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Once developed the executive summary and the background materials should be made 
available to the institutional community as well as the external stakeholders. If possible, establish 
a directory on the mainframe where everyone has access. If possible, establish a link with the 
World Wide Web, Disseminate the information verbally, in writing, and through the computer; 
provide updates through your networks, generate executive summaries for specific "hot topics" 
and distribute to appropriate committees. Networks are a two-way street. People are more willing 
to share if they get something in return. 

The final step is to develop strategies in incorporate process into the organizational 
culture. The first phase is to identify the office or offices that are responsible for developing the 
accountability, assessment/evaluation and planning documents needed for the information model. 
As documents are distributed to the appropriate offices, a copy would be entered on the 
mainframe in the appropriate subdirectory. Documents on the mainframe should be categorized 
into subdirectories based on the sections in the information model for the executive summary. 
The executive summary should be reviewed and updated as appropriate and when it has been 
updated all institutional constituencies should be made aware of its update by making 
announcements at the President's Executive Committee. Council of Deans, and Department Chair 
meetings, through the Web, through the computer networks and the committee networks. After 
the initial executive s^mary has been established, the university community should be surveyed 
as to what other information would be helpful to them In addressing internal and external 
Information needs. Examples of surveys are included. 
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INFTTAL DOCUMEhfT SURVEY 
The Office of Institutional Research and Planning is developing a resource center on the mainframe which will 
contain the university's planning, evaluation, and accountability documents. The plan is for the resource center to be 
a viable and useful service to you, the rest of the university community and the external agencies and/or Individuals 
who are interested in FAU. 

Please indicate on the checklist below the documents you have. Please add any others you have which you 
feel will be of assistance. IF POSSIBLE, please enclose a copy of the documents, or a diskette with the documents, 
or plaro the documents on the N: drive In the directory RESOURCE (N:\RESOURCE). 



Document 


Do you have 
this? 


Academic Program Plan 


Yes No 


Facilities Plan 


Yes No 


Enrollment Plan 


Yes No 


Other State mandated plans 


Yes No 


Strategic, 
long-rangei 

short-range planning documents 


Yes No 
Yes no 
Yes No 


Operational plans for Implementation 


Yes No 


Assessment/Evaluation Plan 


Yes No 


Assessment surveys (1. e. entry students, non-returning, exit Interviews, graduate-senior 
sun^eys, needs assessments, employers, alumni, community needs, admission) 

Pieasa specify or provide copies of the ones conducted within your area 


Yes No 


State accountability documents 


Yes No 


University accountability documents 


Yes No 


115— 

Program reviews * ^ 

Program self-studies for accreditation 


Yes No 
Yes . * No 


Annual reports 


Yes No 


Budgeting documents 


Yes No 


New Program Proposals/Feasibility Studies 


Yes No 


Student sen/ice documents 

(major information, prograrn requirements, admissions, ]ob placement, advising, etc.) 
Please specify or provide copies of the ones which are available. 


Yes No 


Other 

Please specify or provide copies of the ones which are available. 


Yes No 



THANK YOU FOR HELPING US TO SERVE YOU 
Please return to the Office of Institutional Research and Planning 
Computer Center, Building 22, Room 172 
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